Methods and arrangements relating to signal rich art

ABSTRACT

Art can be infused with network identifiers at the time of its creation, rather than as a post-process. The identifiers may be encoded as overt elements of the art, and enable the artist to reprogram—as over time—an augmentation of the artwork experience via network-delivered components. These network components can include stimuli present when the artist created the work (e.g., music), commentary by the artist, video and augmented reality features, audience-crowdsourced content (e.g., imagery of, or feedback provided by, other viewers encountering the artwork), etc. The artwork augmentation can vary with the user&#39;s context (e.g., location, demographics, interests, history). Physical brushes can be equipped to insert such identifiers in graphic arts; other authoring tools are disclosed as well. The network experience can be delivered via smartphones, projectors, and other devices. A great number of other features and arrangements are also detailed.

RELATED APPLICATION DATA

This application is a continuation of application Ser. No. 13/007,372,filed Jan. 14, 2011 (now U.S. Pat. No. 9,563,970), which claims priorityfrom provisional applications 61/295,647, filed Jan. 15, 2010;61/299,270, filed Jan. 28, 2010; 61/303,633, filed Feb. 11, 2010; and61/424,564, filed Dec. 17, 2010.

FIELD OF TECHNOLOGY

The present specification concerns technologies that aid in the creationand utilization of signal rich art (“SRA”)—art having a networkdimension.

BACKGROUND

Artwork tends to be an abstraction of artistic vision applied to fantasyor reality, frozen in a moment. Think of a painting, or song, or movie.The present technologies concern tools for artists and creativeprofessionals allowing creation and delivery of more holisticexperiences to audiences, across multiple dimensions including time,space, media, materials, and senses. Through these tools, art willbecome less discrete, and more continuous, engaging, and collaborative.

Crude notions akin to certain aspects of the present technology havebeen presaged to some extent by posters and magazine ads employingbarcodes, digital watermarks, or image recognition technologies to linkto websites, including Digimarc's work detailed in U.S. Pat. Nos.6,947,571, 7,003,731 and 7,206,820. FIGS. 1A and 1B show some examplesof other artwork employing signaling technologies (barcodes). However,such works are not understood to be created with the tools detailedherein, or have the later-described attributes.

Other related work is detailed in Digimarc's application Ser. No.12/271,772, filed Nov. 14, 2008 (published as 20100119208); 61/157,153,filed Mar. 3, 2009; and Ser. No. 12/490,980, filed Jun. 24, 2009(published as 20100205628). However, those applications focus more ondetection of signals from television content, and related applications.The present application, in contrast, more concerns tools used increating content.

As a programming expedient, some graphics software may assign internalidentifiers to different elements of an artwork (e.g., layers, andpossibly even individual brush strokes). However, such identifiers areunderstood to be for use by the software only in connection with its ownoperations; such identifiers are not accessible, e.g., for augmentingthe experience of audiences via digital devices and networks.

Certain embodiments of the present technology can be used to encodedigital signals into artwork to create a network dimension that may beapplied to all major forms of artistic expression, including graphicarts, music, songs, multi-media, etc. Through use of these tools,digital signals become a novel addition to the artist's palette. Thesesignals can add significant aesthetic and commercial value toconventional works of art. For convenience, the detailed descriptionfocuses on graphic arts.

The foregoing and other features and advantages of the presenttechnology will be more readily apparent from the following detaileddescription, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B show prior art barcodes stylized as art.

FIG. 2 is a block diagram showing certain aspects of one embodiment ofthe present technology.

FIG. 3 shows a graphical interface used by Adobe Photoshop, which can beadapted in accordance with aspects of the present technology.

FIG. 3A shows an excerpt of a palette control used in the FIG. 3 UI.

FIG. 3B shows details of an exemplary user interface by which differentpalette signals can be associated with different response data.

FIG. 3C shows a portion of a system employing dashboard software.

FIG. 4 is a block diagram showing certain aspects of another embodimentof the present technology.

FIG. 5 is a flow chart showing certain aspects of one embodiment.

FIG. 6A is a top-down view showing, for four vertical zones A-D of adisplay screen, how more distant parts of the screen subtend smallerangles for a viewer.

FIG. 6B shows how the phenomenon of FIG. 6A can be redressed, bypre-distorting the information presented on the screen.

FIG. 7 shows how signals in art can distorted in vertical and horizontaldimensions, in accordance with position of a viewer.

FIGS. 8A and 8B show a paintbrush employing certain principles of thepresent technology.

FIGS. 9A and 9B show brush strokes that may be created with the brush ofFIG. 8.

FIG. 10 shows a brush stroke modulated in accordance with aninformation-conveying code.

FIGS. 11A and 11B show another paintbrush employing certain principlesof the present technology.

FIG. 12 is a block diagram of an illustrative system using aposition-based encoding arrangement.

DESCRIPTION

In accordance with one aspect of the present technology, a contentauthoring system—such as a computer system running Adobe's Photoshop CS4Extended software—is modified to enable inclusion of network attributeswith art, at the time the artwork is created by the user.

FIG. 2 gives an overview of one particular arrangement. A computersystem executes a graphics software package for authoring artwork. Inthe authoring process, identifiers are associated with individualelements of the artwork (or the artwork as a whole). A remote serverincludes a database that associates each different identifier with dataindicating one or more corresponding network responses. These responsesmay be regarded as “applications” that are triggered by user discoveryof corresponding features in the artwork.

When artwork produced by such an arrangement (e.g., a product label, oran electronic billboard display) is imaged by a sensor-equipped device(e.g., a smartphone), the identifier is discerned from the artwork. Itis then passed to the database, to retrieve the associatedresponse/application information. The device then acts in accordancewith the application information, such as by loading a page from theWorld Wide Web, executing a Java applet, rendering a video, connectingto a social network, initiating an augmented reality experience, etc.

This response application may, for example, electronically link a personviewing the artwork to further information or other content associatedwith the artwork, e.g., hearing the artist's spoken commentary about theart, or reviewing comments left by previous viewers. A family'sChristmas card can have network extensions that, when viewed through asmartphone, cause a video recording of the family expressing theirgreetings to be rendered. The responsive applications can be varied overtime. Responses can also vary based on contest, such as geographicallocation of the art, the viewer's age and language, etc. Commercialopportunities are numerous.

Adding a network attribute to art at the creation process can beanalogized to a painter's technological palette—a resource from whichthe artist can draw in composing an artistic creation. In this case, theartistic creation has one or more network attributes.

Network attributes can be of many forms. Some take the form ofidentifiers associated with artworks, or components thereof. (Theidentifier may be a binary number of any length, e.g., 4 bits, 12 bits,36 bits, 96 bits, 256 bits, etc., or it may be another form of datasymbol—not necessarily binary.) The identifier can be associated—in alocal or remote data structure (e.g., a database) with relatedapplication information, such as a network address for linking, orstored computer code for execution, or a multimedia file for rendering,etc.

One way to convey an identifier with artwork is by use of a paintbrushtool that lays down pixels (or defines a vector graphic) having apatterned or textured arrangement that encodes or otherwise conveys theidentifier. The patterning may be so subtle as to be imperceptible tohuman viewers (e.g., steganography, or digital watermarking), or it canbe strong and overt. This latter case includes digital watermarkingtechniques in which the amplitude of the encoded signal is set to aplainly visible level. It also includes brushes that lay down shapesfilled with machine readable symbologies, such as 1D or 2D barcodeelements, or data glyphs. For expository convenience, all such signalingtechniques are referenced simply as “digital signals.”

Such signals can be within borders of a shape laid down by the tool orin an associated effect, such as shadow or 3D elevation.

A given signal can also span several graphical elements, such as digitalwatermark line width modulation of a set of lines (e.g., in ornamentalguilloche patterns), and lines comprising barcodes.

In certain arrangements, artwork employs digital signaling as a desiredfeature—rather than an attribute to be hidden or obscured. What, in manyprior contexts, may have been regarded as undesirable “noise,” isinstead exploited as an element of artistic creation. Just as art can beformed from assemblages of utilitarian articles, so too can art beformed using expressions of signals as component parts.

Some of Andy Warhol's most memorable works are those in which heemphasized mechanical aspects of graphical arts, such as enlargedhalftone dots and spatially separated color screens—bringing to the forefeatures that normally escape viewers' attention. In like fashion,artisans employing the present technology may use data-rich patterns andtextures in dominant roles. (While such abstraction is not common inrepresentational imagery, much artwork is not representational. Considerfamiliar art forms such as upholstery, rugs, and clothing fabricdesigns.)

In an exemplary embodiment, the signaling pattern applied by the brushis selected from a palette. The palette may offer different signal formsbetween which the artist can select by interaction with a graphical orother user interface. A mosaic of signal forms can be employed in aconcerted fashion to create complex network attributes.

FIG. 3 shows an illustrative graphical user interface 20. Included is adialog box 22 from which the user can select brushes to lay downgraphical elements, such as brush stroke 24. Also shown is a palette 26.Among the different effects (e.g., colors, textures) selectable from thepalette 26 are effects that additionally convey plural-bit identifiers.

FIG. 3A is an excerpt of a palette control, from which the artist canselect different effects to be laid onto the canvas. Some aresignal-conveying; others are not. In one particular arrangement, eachtime the artist dips a brush in a new signal-conveying palette box, anew layer is automatically added to the artwork, on which the new signalis applied. The artist contemporaneously or later associates the signalwith metadata and applications, such as by a campaign manager dashboard.The campaign manager enables the artist or her associates to program theexperience of the audience and the feedback mechanisms triggered by thefocal artistic embodiment (e.g. photo, ad, video, song).

The campaign manager dashboard can be modeled after other softwarecontrol dashboards, such as are used to manage marketing or web videocampaigns. An illustrative dashboard may include user interface controlsby which the operator can perform tasks associated with: (1) contentproduction; (2) distribution (publishing); (3) engaging the user withfeatures that invite user involvement; (4) analysis; and (5)monetization. FIG. 3C shows a portion of such a system.

Controls associated with producing the content allow the operator tocraft the work and program the audience experience (e.g., drawing fromdifferent content sources, and employing various applications andfeedback mechanisms), as well as manage same (e.g., adding and editingmetadata, etc.).

The content production controls can provide, or access, libraries ofcomponent works, including music and videos. These may be accessed notjust by bibliographic details, but also by genre, keywords, recommendersystems, etc. For example, an artist working on a print ad for boots mayinput keywords (e.g., by typing, speech recognition, etc.) that describeor relate to the subject and its targeted audience. The dashboard mayquery external resources and libraries, and make suggestions about (andprovide links to) soundtracks and other content that might be suitablefor association with the artwork (e.g., songs such as “These Boots areMade for Walkin” by Johnny Cash/Nancy Sinatra/Jessica Simpson/GeriHalliwell/Jewel/Megadeth; and “Boots On” by Randy Houser; stylizedpatterns of boot treads; archival WWII movie footage showing muddyboots; etc.). In like fashion, some music recommendation engines work ondescriptors such as “edgy,” “dangerous,” “outdoors,” etc., and suggestmusic tracks that connote the indicated attributes. Certain initialsuggestions may be provided to a recommendation engine (e.g., Apple'sGenius technology) to identify still further content that may also besuitable. Content production tools providing such functionality thusextend the artist's creative process, and can enhance the resultingwork.

Distribution controls allow the artist, or agent, to configure delivery(e.g., across various channels and to various platforms), and also totailor times and context in which the context is to be presented.

User engagement controls enable the operator to employ a variety oftechnologies—often based on social networks—to involve the audience withthe content, such as inviting user feedback (e.g., uploading usergenerated content, chat, comments, ratings, blogging, etc.) andfacilitating user participation in other contexts (e.g., Twitter, RSSfeeds, SMS notifications). Just as the artist may complement a visualartwork with a congruent soundtrack, so too may the artist complementthe artwork with one or more social networking features that areconceived to complement the work.

Some dashboards may include extensive analytic functionality. These caninclude spreadsheet and charting options that allow analysis of contentdistribution by time/location/channel/etc., audience demographics,length of viewer engagement, click-through statistics for differentcommercial and non-commercial links, etc. etc.

The monetization aspects of the dashboard typically involve integrationwith different ad platforms and formats (e.g., Google, DoubleClick),commercial sponsorships, e-commerce activities, etc.

In some respects, use of such as system is akin to production of amotion picture. In motion picture production, individual scenes of amovie may initially be captured in front of a blue screen. Later, inpost-production, digital artwork is substituted for the blue background,sound effects are added, and computer graphics effects are inserted andintegrated. These individual scenes are then stitched together withcertain transition effects, etc. In like fashion, an operator of theillustrated system assembles an artistic production in stages—firstcreating elemental parts of the work, and then defining and addingfurther aspects of the work using the various controls and optionsprovided by the software tools. (In many cases, the elemental partsinclude signal attributes with which respective applications are laterassociated.)

One artist may use the dashboard to tailor a work so that differentresponses are triggered, depending on which model of cell phone a vieweris using. Another artist may tailor a work to respond differently basedon the age or gender of the viewer, etc., etc.

One artist may specify Amazon as a destination for interactions; anothermay specify Ebay.

One artist may use the dashboard to establish a FaceBook page associatedwith the work; another may use the same dashboard but instead associatethe work with a MySpace page.

One artist may use the dashboard to associate the artwork with trackingservices provided by Omniture (Adobe); another may use the dashboard toselect DoubleClick (Google). Through the dashboard the artist may selecta free package of analytics, or a $500 package. Etc., etc. After theartist has configured the work in the desired fashion, she may send theassociated data (e.g., a file defining a print ad) to a magazine forinclusion in the next issue. When audience members interact with theprint ad using their cell phones, tracking/analytic data is thenreceived back (by the artist or other designee) from theearlier-identified service provider, in accordance with theearlier-selected package of services. Etc.

Thus, through implementations of the present technology, an artist canknit together a creative aesthetic that transcends the traditionalnotion of an artwork as being a singular object in time and space, andshifts the conception of an artwork to being a digitally orchestratednetwork of complementary features, senses, and social experiences.

In some arrangements, the artist can associate digital signals withmetadata and applications using functionality built into asuitably-equipped version of Photoshop-like software. For example, theuser interface (of which the palette 26 of FIG. 3A may be a part) caninclude a control 30 for associating different digital signalsrepresented in the palette with different applications. For example, asshown by the dashed arrows in FIG. 3B, the interface may allow the userto paint pattern from the palette into boxes 32 associated withdifferent types of applications, e.g., web links, Java code, music, etc.When one of the boxes is painted with a palette pattern, a further inputcontrol 34 appears, in which the artist types details of the desiredapplication to be associated with that pattern (e.g., by web address,MP3 name, executable software identifier, etc.).

Each time one of the boxes 32 is painted, a further box appears underthe same category, allowing the user to define multiple signalassociations of each type, if desired.

In one viewing mode, the authoring software overlays textual annotations(e.g., A1, C1, etc.) on regions of artwork that have been painted withdifferent digital signals, to allow the artist to readily identify theapplication associated with each region.

In one embodiment, the particular numeric (or alphanumeric) identifierconveyed by a graphical pattern is assigned by the software, in anautomated fashion (e.g., sequentially through a predefined numberingspace, randomly, etc.). However, another graphical user interface allowsthe user, if desired, to enter a particular identifier that the patternis to convey.

In another embodiment, when the user selects an information-conveyingeffect from a palette, a dialog box appears requesting the desiredidentifier. The user can type-in the identifier, in hex, decimal orotherwise, or can specify the identifier by manipulating a graphicalcontrol, such as a slider bar. If the user hits Enter without enteringan identifier, the system assigns one on its own.

In yet another embodiment, the user correlates the identifier withcertain metadata, contemplating triggering of services associated withthe metadata when the identifier is recognized by a digital devicesensor.

Many image file formats (including most of those supported by AdobePhotoshop) store metadata in association with the artwork. Much of themetadata is produced by the software during the authoring process, andis stored without involvement of the user. This data can relate to theartwork (e.g., canvas size), the artist (e.g., biographical details),the circumstances of artwork creation (e.g., dates and times worked on;location where created), etc. The metadata storage can be used forstoring information associated with the present technology. In someembodiments, it may be the source from which the external databasemirrors relevant information.

The metadata produced by the software can be incorporated into thedefinition, or characterization, of audience augmentation/response. Forexample, when a viewer encounters a print advertisement, localizationdata matched with contextual data can shape the targeted response. Theresponse can further depend on other metadata associated with theartwork. Moreover, audience responses to artwork, or to applicationstriggered by interaction with the artwork, may be logged in the storageas further metadata—helping shape interactions with future audiences.

Brushes are not the only tool that can be used to apply signal-richeffects to artwork. Essentially any tool offered by the program can beso-employed (several others are shown in control 28). For example, agradient fill tool can be used to apply a signaling pattern to aparticular layer or region of the artwork.

Moreover, the tools needn't additively combine signal to image elements.The encoding can also be effected by subtracting an encoding patternfrom another element, region or layer. The encoding can also be effectedby changing the mode of the artwork, such as by choosing a bit-mappedrepresentation using a dither that conveys a data payload (glyphs areone example). A half tone or other screen effect can likewise impart adigital signal.

In still other cases, the program can algorithmically change an objectdepiction in subtle respects to represent data. Consider a face.Dimensional statics for human faces are well categorized (e.g., for usein facial recognition applications). Typical distances between pupils,between mouth corners, eye midline-to-nose, etc., and ratiostherebetween are well understood. On recognizing a face in a drawing orartwork, the program may slightly modulate (adjust) certain of thesestatistics—akin to snapping vector graphic elements to grid points—inorder to represent one of several different plural-symbol payloads.

Reference was earlier made to association between an identifier conveyedby the artwork, and other functionality (e.g., associated in adatabase). The graphics editing program (e.g., Photoshop or GIMP) canprovide a user interface by which this association can be defined andstored, either locally, or in a cloud database.

One such interface is a dialog box that is tabular in form, populated onthe left with identifiers used in the artwork, and on the right withblanks in which the user can type, or drag, web links.

In another arrangement, the identifiers are not revealed to the user.Instead, the user simply right-clicks on a layer, brush stroke, or otherdata-conveying element. The program responds with a dialog box (perhapsafter selecting an option from a drop-down menu) inviting the user todefine a network resource that is to be associated with the identifiedelement. In response to such user action, the software submits data to adatabase associating the indicated network resource with an identifierfor the element.

In still other arrangements, the palette has paired identifiers andassociated network attributes, both pre-existing at the time the paletteis used to create a work. For example, the software program can presenta palette of audio effects—sounds of nature, Rolling Stones' I Can't GetNo Satisfaction, the opening notes of a Beethoven symphony, etc. Anartist can touch a brush to the desired palette tile, and then strokemusic across different regions of the artwork. (This is typically donein a “layer” distinct from visual elements of the art, simply as a dataorganization expedient.) When these regions of the artwork are laterimaged by a consumer, the selected music is identified through adatabase lookup, and rendered.

In another particular implementation, the artist listens to music as shecomposes an artwork, and may elect to have such music associated withthe artwork. For example, an MP3 player (hardware or software) canoutput meta data indicating the song currently playing. This informationcan be associated with a particular tile in a palette that the artistcan select. Or it can be associated with the entire artwork, withoutresort to tool and palette.

Alternatively, the computer system can include, or make use of, musicrecognition technology, such as is described in Shazam's U.S. Pat. Nos.6,990,453 and 7,359,889, and its published patent applications20050177372 and 20080154401. For example, a microphone coupled to thecomputer can sample ambient audio in the artist's environment.Fingerprint information is then derived from the sampled audio, andsubmitted to a fingerprint database to identify the song. Thisinformation is then associated with one or more identifiers conveyed bythe artistic work. When a viewer encounters the work, the songidentification can be retrieved from the artwork, and used to obtain acopy of the song. The song can then be rendered to the viewer whilestudying the art—allowing the viewer to perceive the art in the audioenvironment in which it was created.

Such an arrangement is shown in FIG. 4, including a microphone 42coupled to the artist's computer system, and a speaker or other soundtransducer 44 associated with the viewer's smartphone.

Just as such an arrangement can allow the viewer to experience theartist's auditory environment, similar principles can allow the viewerto experience other aspects of the artist's creative milieu. Forexample, the artist's computer system may have a camera that producesstill or motion imagery of objects and scenes in the artist's view.(Such a camera may be borne by the artist, e.g., using a head-mountedarrangement.) This image data can be stored in a database, andassociated with the art, for later recall by viewing devices. Or, imagedobjects and scenes can be recognized using fingerprint, SIFT, and/orother visual recognition techniques—allowing metadata identifying suchsurroundings to be identified and stored in association with theartwork. Again, a viewer device can extract signals from the art,allowing retrieval of information about the artist's visual environment.Again, renderings of such environmental features can be based on dataobtained from the web, or otherwise, and provided to the viewer whileexperiencing the art

Other sensations, such as smell, vibration, touch, etc., derived duringthe creation process (or otherwise) can likewise be associated with theartist's creative work. For example, haptic devices can be employed bythe public to emulate artist movements associated with the work, givingthe audience a further dimension of artistic experience.

Moreover, data from and relating to sensors involved in the creation ofart can be stored for use in later network associations: the pressureapplied to a Wacom stylus, the speed of a stylus stroke, the particularbrush used—all of these data can be associated with artwork (the presentwork, or others). They can serve as APIs or hooks that are grouped andexposed to augment the art. When listening to the Rolling Stones, anartist may use broad brushes with heavy pressure. These attributes canbe grouped together and associated with a particular audio experience

Because network responses can be associated with different elements ofthe artwork, a viewer may be provided different experiences by focusingthe sensing device (e.g., smartphone) on different regions of the art.Focusing on a mountain depicted in an artwork may evoke sounds of windsscreaming around an exposed peak; focusing on a valley in the same piececan summon audio renderings of a brook.

In addition to varying with location, the associations can also varywith time, e.g., tracking the time spent by the artist in creating awork. One example is the artist's auditory environment being renderedfor the viewer during a viewing interval. The longer a person views theart, the more the auditory information is rendered.

In some arrangements, the editing software (or other software) iscapable of “replaying” a work of art to a viewer through its process ofcreation—from a blank canvas (screen) to a finished work. That is, thefinished artwork file can include not only information about theplacement of each artwork element, but also temporal information aboutthe order and timing by which they were applied. Thus, in addition toperceiving the finished work, a viewer can also use a viewing device(and perhaps others) to explore the process by which the artwork wascreated. The various network associations can then be rendered withtemporal ordering and intervals corresponding to the timing by which theartwork was created. (A time warp control, such as a shuttle, may beprovided in the rendering UI, to allow the viewer to experience thecreative process at an accelerated rate, so that an artistic processthat took hours or days can be rendered in minutes or seconds.)

In some arrangements, the artwork file (e.g., an Adobe *.PSD file) caninclude stored timecode references—a clock with which different parts ofthe creative process are timed. Just as network actions can beassociated with visual elements of an artwork, they can likewise beassociated with timepoints in the chronology of the artwork's creation.

For example, a person studying an artwork for ten minutes may beprovided a comment, or audio clip, associated with the tenth minute ofthe work's creation. (Again, with a time warp control, the networkextension associated with a ten minute timecode can be activated, e.g.,after one minute of viewing.) The artist can also associate networkfeatures of art with desired timepoints—regardless of timing of thecreation process.

One way to implement such visual feature-independent timecodeassociations is shown in FIG. 5. The remote system that resolves networkassociations for the artwork starts a timer when a user's smartphonefirst engages with an artwork (e.g., by decoding a signal embedded inthe artwork, and sending it for network resolution). The remote systemthen loops—checking to see if there is any action associated in thedatabase with the current timer count. If there is an association (e.g.,a sound clip is to be rendered, or a word is to be presented), thesystem next checks whether the user is still engaged with that artwork.

User engagement can be tested in various fashions. For example, if thesmartphone publishes its location (e.g., data from a GPS receivermodule) in a manner accessible to the remote system, then movement ofthe user away from the artwork's vicinity (e.g., the location from whichthe first engagement was sensed) can indicate disengagement. Anothertest is whether the same smartphone has engaged with a different pieceof art through the remote system, in which case disengagement with theformer artwork is then indicated. Still another test is whether thesmartphone has recently (e.g., in the past 15 second seconds) discernedand sent a further identifier associated with the artwork to the remotesystem for resolution. If so, this indicates the user is still engaged.

If the user is still engaged, the remote system then acts on thetime-associated response, such as by providing a link or content to theuser, or performing other action.

The process next checks whether there are further time-based actions yetto be acted on for that user. If not, the process terminates. Otherwise,the timer is incremented and the loop repeats. (The timer increment canbe set to suit the needs of a particular application. For some, agranularity of one second may be appropriate; for others, one minute maybe better. This is indicated by the programmed “Pause” in FIG. 5.)

In some cases, a network response may have its own programmed temporalaspects. For example, a Flash presentation may include its own scriptingtimeline. Once this response is initiated, it may play out automaticallyin accordance with its programmed sequence. Such a presentation caninclude still imagery, audio, video, etc. In such cases, the remotesystem needn't orchestrate the timing of responses; once the Flashsequence is initiated, it controls the time-based actions.

It will be recognized that an arrangement like that just-described, andillustrated in FIG. 5, can likewise be effected on the user's smartphone(the “client” side). That is, when the phone first engages with anartwork, it can download information about all of the time-relatedresponses from the remote server. A phone timer can then indicatetimepoints at which certain time-related actions should be taken, andtrigger same.

Audience members may also take away snippets from the experience, suchas a ringtone, an image (e.g., for desktop wallpaper), a coupon, an RSSfeed, etc., so as to let the experience live on with the audience afterthe initial “performance.” In some circumstances, the user's smart phoneis the medium by which such snippet is conveyed from the artisticencounter. In other circumstances, other means of delivery can beemployed. The responsive application can bookmark a web site to whichonly individuals that have encountered the artwork are granted access(e.g., by a URL and associated key that are transferred to the phone).

Database associations between artwork elements and network responsesneedn't be defined at the time the art is created. These associationscan be defined later—either using the original artwork creation tool, oranother database editor. Similarly, associations once-made needn't bemaintained. They can be later changed—usually by the artist, but also insome instances by viewers.

Some artists may be frustrated that the public doesn't “get” their work.After a work has been published without the understanding or acclaim theartist desires, she can associate further information with thework—explaining its significance, imagery, etc. New viewers, aided bythe artist's commentary linked from the work, may finally “get it.”

Some artists may offer different narratives for their work, betweenwhich a viewer can select (or the viewer may review all). One narrativemay be for school age children, another for adults. One may be forpeople who express an appreciation for the work, another for those whoexpress a distaste for it. Etc. The artist's associated informationneedn't be auditory; it can comprise images or words calculated totrigger new insight into understanding the artwork.

In some instances, aspects of the viewers' experiences are added to theart's network dimensions. For example, an artwork installation, or acommercial electronic signboard, can include a camera system to sensethe number of viewers, their genders and approximate ages, and the timethey spend viewing the art. (Such arrangements are known; e.g.,published patent application WO 2007/120686 discloses electronicbillboards equipped with camera systems that sense viewers and estimatetheir ages and genders.) One of the network features associated with anartwork can be the demographics, and time, that others have spentviewing the art. In an augmented reality (AR) smartphone viewer, forexample, the AR software can identify the artwork by signals it conveys,look up a corresponding network resource that includes viewerdemographics, and then present to the user a bar graph overlay showingthe cumulative time spent by people of different demographic groupsviewing the art. A new viewer can scan an artwork gallery, and quicklyidentify the works that have commanded the most, and least,attention—optionally broken down demographically. (Ditto billboards andelectronic displays in subway stations, etc.)

More than just statistics can be collected and rendered in this manner.With appropriate privacy safeguards or permissions, a camera system thatviews the viewers can post their images to a network resource associatedwith an item of art. A viewer of the artwork can then view previousviewers—the familiar pastime of people-watching, this time through thelens of an artwork of shared interest. (The artist can likewise view anaudience that her work attracts.)

Again, augmented reality arrangements can also be employed to vary thevisual appearance of the artwork, as viewed by a smartphone or the like.For example, superimposed on the artwork may be silhouettes, ortransparent ghost images, of the last ten people who lingered in frontof the artwork for more than ten seconds, or the last six children, etc.

Similarly, people can provide feedback, e.g., spoken, by text messaging,or otherwise, responsive to art. (An art installation, such as anelectronic display terminal, can include a microphone to sense spokenfeedback, or same can be called-in using microphones in users'smartphones. Network links associated with an artwork can presentviewers with a destination number to which text messages can be sent, ormay provide a web-based UI for viewer text entry.) Artworks may haveFaceBook pages, Twitter sites, or other social network presences,through which interactions are channeled. All user feedback, too, can bestored in association with artwork identifiers—available for recall byother viewers, and the artist. The artist, in turn, may take thiscollected feedback and use it as fodder for the creation of an entirelynew work.

A new modernism may emerge, in which the “Expose Yourself to Art” slogantakes on new meanings.

It will be recognized that such conceptions of art take on a socialaspect, through which the artist can draw from crowd-sourcingtechnologies to augment the experience, adding evolutionary aspects toan artwork.

Consider the cult-following of the movie Blade Runner, which has led tocreation of the notion—unintended by the movie's creators—that HarrisonFord's character is actually an android. An artwork's viewers maysimilarly continue and extend the artistic tradition. That is, acommunity of viewers can respond to, and may contribute to, artwork inunexpected ways—perhaps causing the meaning, relevance or import of theart to evolve over time.

In turn, an artist may reprogram the network appurtenances of anartwork, in a manner akin to a live performance—responding to theaudience's responses. As the audience response changes further, theartist's sur-response changes further, etc. An episodic experience canensue.

Alternatively, the artist may surrender the network content of anartwork to the audience—perhaps sometimes acting as a moderator, perhapsnot. Perhaps the artist returns five years later to find the networkpresence of the artwork wholly different than his originalinstantiation.

Use of smartphones and other processing devices to mediate and augmentinteraction with artwork enables further possibilities. One is theconcept of Easter eggs—messages or features exposed only after an arcaneor unusual set of events.

Some Easter eggs may be associated with obscure features in an artwork,which most viewers will overlook or not network-explore. For example, ifa viewer focuses the smartphone on a particular dandelion in a field ofdepicted dandelions, a surprise response may be revealed. Some suchfeatures may not be exposed for years, if ever.

Another artist may feel that until a person has viewed an entireportfolio of associated works (e.g., 20 prints, distributed acrossseveral galleries in town), they cannot really understand the artist'smessage. The artist may arrange to deliver a bonus presentation (e.g., aone-time only rendering of a video showing the artist winking at theviewer) to anyone who views all such works.

Each time a smartphone queries the database, the database can log anidentifier of the smartphone (e.g., a device ID, a MAC address, a phonenumber, etc.), together with information about the record(s) it queries.The database can then determine when a given smartphone has submittedqueries relating to all works in a portfolio. When the last query issubmitted, the database system can return the Easter egg presentation tothe surprised viewer.

By arrangements like the foregoing, an artist can craft or influence thepublic's experience of artwork, leading to a richer interaction with theartist's motivation or message. Even a static print can become a portalto experiences that can be akin to performance art.

Naturally, such technology has commercial implications as well,extending to advertising, product packaging, video entertainment, etc.Even art galleries dabble in commerce. A gallery may be equipped withcameras, Bluetooth receivers, or other sensors (e.g., video cameras) bywhich a person's temporal engagement with a particular artwork can besensed and assessed. Likewise, the person's interactions with theartwork's network extensions can be noted. If a gallery patron spends 45minutes considering an artwork, or returns to it repeatedly, the gallerymay want to follow-up to complete a sale. Network traffic can bedata-mined to reveal what patrons are interested in what artists,genres, etc. Follow-up mailings or other targeted marketing efforts canbe based on such information.

While described in the context of authoring digital still image artwork,the same arrangements can likewise be applied in composing other media,such as audio, audio-visual works, sculptures and electronic 3D media,etc. Moreover, it will be recognized that still image editing tools alsoplay a role in creation of many other content types, such as painting 3Dmodels and wrapping images around 3D shapes used in film and video, andauthoring special effects and animations. Likewise, the tools andtechniques disclosed for still image artwork are equally applicable incontexts such as web page design, and authoring network rich documentsuseful in fields such as engineering, science, architecture andmanufacturing.

Education is another field in which the detailed technologies areadvantageous. Media that engages individuals to explore and interactwith instructional materials fosters a more captivating and effectivelearning experience than passive teaching aids. Likewise with printmedia serving other communication roles.

The technologies described herein can be rendered in paintings,sculpture, and other art forms where the use of computer-aided designheretofore has been nominal or not used, by employing emulation. Forinstance, a robotic arm can apply oil paints by palette knife to acanvas as the artist paints or based on subsequent analysis of theprocess of the painting or of the artwork itself. A CNC 3D millingmachine can create a shaped polymer medium on which paints or inks canthereafter be deposited, mimicking the 3D effects of oil painting.Graphics editing software can include provisions for defining the 3Dtopology, or brush strokes, comprising the artwork. In some instances,the information-carrying is effected, in whole or part, by the 3Daspects of the art.

While the foregoing disclosure focused on machine-readable symbologies,digital signaling can be effected otherwise. Some techniques aregrounded in computer vision (object recognition) techniques. Forexample, one way to associate an identifier with artwork is throughimage fingerprinting. Attributes of an artwork, or a portion thereof,are distilled into a set of numbers, or features, which are stored in adatabase, and later matched against unknown works to identify same. Thefingerprint data serves as the identifier. (Image and videofingerprinting techniques are detailed in patent publications U.S. Pat.No. 7,020,304 (Digimarc), U.S. Pat. No. 7,486,827 (Seiko-Epson),20070253594 (Vobile), 20080317278 (Thomson), and 20020044659 (NEC).)

A particular form of fingerprinting is scale invariant featuretransforms (SIFT), as detailed in Lowe, “Distinctive Image Features fromScale-Invariant Keypoints,” International Journal of Computer Vision,60, 2 (2004), pp. 91-110; Lowe, “Object Recognition from LocalScale-Invariant Features,” International Conference on Computer Vision,Corfu, Greece (September 1999), pp. 1150-1157, and in U.S. Pat. No.6,711,293. Again, SIFT data derived from imagery can be stored in adatabase, and used as a digital signal by which a visual work (orportion) can be recognized.

In the prior art, images are authored (and music is composed/performed)without regard to recognizability through automated techniques, such asfingerprinting or SIFT. However, some artworks are much more suitable tounambiguous identification by such techniques than others.

In accordance with another aspect of the present technology, software isalert to the recognizability of imagery and image excerpts (or sound andsound excerpts) through one or more recognition algorithms (such asthose in common use by smartphones, e.g., SIFT and Shazam). If a pieceauthored by an artist is found to be marginal, or deficient, in terms ofits recognizability, the software can undertake one or more actions. Oneaction is to alert the artist, such as by a warning message presented ina dialog box on the computer screen. Tips to enhance recognizability maybe provided in the message. Another action is to automatically adjustthe artwork, within certain permitted ranges, to enhancerecognizability.

To illustrate with an improbably simple example from the 1D world ofaudio recognition (the image analog, in 2D, is a bit more complex),consider an audio fingerprinting algorithm that performs an FFT on a 10second excerpt of music, low-pass filtered at 3 KHz. The audio energyresulting from the analysis is binned into 10 frequency ranges, Bin1encompasses 0-300 Hz, Bin2 encompasses 301-600 Hz, Bin3 encompasses601-900 Hz, Bin4 encompasses 901-1200 Hz, etc. The audio fingerprintcomprises a four number ordered sequence, identifying the four bins withthe most energy, in decreasing order. A particular music excerpt mayhave dominant tones at 750 Hz, followed by 450 Hz, followed by 1000 Hz,followed by 200 Hz. The fingerprint of such an excerpt is thus{3,2,4,1}.

That's well and good, and unambiguous. But what if the third-mostdominant tone was not 1000 Hz, but 1195 Hz? This falls within Bin4. Butit is close to falling within Bin5 (1201-1500 hz). In some distortionscenarios (e.g., sped-up radio airplay), or in noisy sensingenvironments, this element of the fingerprint may be determined to be a“5” instead of a “4.” Misrecognition of the music may result.

Accordingly, during composition of the music (or, in the imageryexample, during authoring of an artwork), such ambiguous cases can beflagged so that remedial action may be taken, either by the artist, orby the programmed system. The artwork can be tweaked in minor respectsto minimize such borderline cases.

Consider the case of SIFT-based object recognition. Basically, SIFT isbased on identification of image features (commonly termed keypoints)that survive blurring, and that form recognizable constellations ofpoints despite scaling and rotation. Feature edges inimages—particularly corners points—are commonly used as SIFT keypoints.

The image editing software can apply SIFT-based analyses to artwork(e.g., periodically as it is being developed, when it is saved, or atthe time of rendering). If an inadequate number of features is found tobe identifiable after blurring, or the features are not satisfactorilyrobust to scaling and rotation, the software can alert the user, andoffer tips for improvement.

Tips may include instructions to increase the local image contrast incertain regions; to apply an edge sharpening filter to other regions,etc. The software can highlight the relevant regions, or outline themusing an image editing tool provided for this purpose (e.g., the Marqueeor Lasso tool) to facilitate the suggested user edits. Or, again, thesoftware can undertake such revisions automatically (perhaps with artistapproval).

In some cases the process is iterative, with a potential deficiencynoted, an adjustment made, and the resulting work again tested for thedeficiency. If it persists, a further adjustment can be made, etc.

Other forms of digital signals can be similarly monitored foreffectiveness. If an artist applies a digital signal-conveying lightwash to a work, the program may alert the artist if the signal is likelytoo subtle to be reliably detected. In some embodiments a bar graph orother UI device can be employed to indicate to the artist the strength(detectability) of the artwork's digital signal(s). If the artistprovides information about the rendering circumstances, the program canmodel how the signaling will be transformed by the rendering operation,and the degree to which the signals will be detectable.

(Desirably, detectability is not an issue, because the artwork mayconspicuously feature human noticeable attributes of the signal. In suchcase, the signal is a dimension of artistic freedom in the work; not anartifact to be suppressed or obscured.)

In some instances, the data payload for an artwork element is not fixedat the time the element is added to a canvas. In such case, aplaceholder payload can be applied, or a texture or other effectsimulating a payload can be used, in representing the work on theartist's computer screen. But this is just an approximation. (Sucharrangement is akin to time codes in works that have a time dimension,or pantone color specification for color works.) When the artist'screative process is finished, and it is time to render the work into afinal form, the software may fix the data payload(s), e.g., by invitingthe artist to provide this information, or by assigning sameautomatically if the artist does not. At this point, the softwaregenerates the patterns and textures that are included in the finaloutput.

In some cases the rendering engine has flexibility in how encodings areexpressed. For example, different renderings may be equivalent to thehuman visual system, but offer different encoding possibilities. Colorpairs are one example (c.f., patent application 20060008112). Another isline patterns; the more lines in a region, the less any individual lineneeds to be altered for line width modulation purposes, etc. Therendering code may alter the work in ways not evident to human viewers,to increase the decodability of one or more associated identifiers.Previous versions of the work, e.g., as viewed by the artist on thescreen, are simply proxies for this final work.

Once a work is rendered in its final digital form, the program cananalyze this output and generate a map showing the artist the differentsignals across the work, and their relative strengths (and in someinstances their respective scales, or optimal reading distances).

It will be recognized that the different forms of identifiers notedabove serve as virtual hooks to which network attributes can attach.Through such hooks, various tags can be applied to artwork—both at thetime of its creation and throughout its life.

Art installations increasingly make use of data projectors—devices thatproject still or moving imagery in accordance with programmed data. Manyof the arrangements detailed in this specification can make use of suchprojectors. For example, in those instances in which augmented realitywas mentioned, the superposition of features with artwork can beaccomplished, publicly, by a projection system. (A projector may also beincluded as a smartphone component, and used in many such instances.) Asin other arrangements, aspects of the artwork can change as a functionof context, such as the particular viewers present, history, time, etc.

Sometimes the scale at which digital signals are sensed by a smartphonecan influence their decodability (e.g., a postage stamp-sized barcodemarking probably will not be readable from a distance of ten feet). Theimage editing program can present a dialog box asking the artist tospecify a nominal viewing distance. (A default value, such as four feet,may be used if the artist provides no information.) The program mayfurther ask about the height or width of the artwork as it is expectedto be finally rendered. (Canvas size is typically specified as part ofan artwork's creation, but this size is sometimes not accuratelyindicated, or the artist may decide to render the artwork at a differentscale.)

Based on the indicated viewing distance and dimension, the digitalsignal (e.g., machine readable encoding) can be scaled to assuredetectability. Desirably, the distance and dimension information isavailable before the graphical elements incorporating such features areapplied by the artist to the canvas (or completed by the software).Knowing these parameters, the encoding can be sized appropriately in thepalette, and the artist can correctly judge its effect in the artwork aspresented on screen. However, if the artist later decides to change,e.g., the scale of the work, the program can automatically adjust thescale of the digital signal to assure detectability under the revisedcircumstances.

The ratio between viewing distance and dimension is usually the keyparameter. (A four foot print viewed from four feet has similardetection attributes as a ten foot print viewed from ten feet.) If oneis changed without the other, a change in the scale of the encoding mayprove beneficial.

In some cases, however, the artist may desire that artwork conveydifferent impressions from different perspectives. Signals in theartwork may serve such purpose. For example, the artwork may incorporatedigital signals at different scales—sensing from one distance yields oneidentifier; sensing from another distance yields another identifier.Similarly, the signals may be geometrically warped in anticipation ofviewing from certain perspectives.

Consider FIG. 6A, which is a top-down view showing how a viewer, atposition 52, perceives an artwork 54, when viewed from near the rightedge.

If the artwork is regarded as having four equal-width verticalquarter-panels A-D, it will be seen that the nearest panel (D) subtendsa 45 degree angle as viewed by the observer. The other quarter-panels C,B and A subtend progressively smaller ranges of the observer's field ofview. (The entire artwork spans about 76 degrees of the observer's fieldof view, so the 45 degree apparent width of the nearest quarter-panel islarger than that of the other three quarter-panels combined.)

In anticipation of this distortion, and to provide a signal to viewerson the right side of the work that is unavailable to viewers, e.g., onthe left side of the work, the image editing software can warp thedigital signal characteristics in accordance with the location of theviewer. FIG. 6B illustrates one form of such distortion. If the artwork54 is again regarded as having four vertical panels, they are now ofdifferent widths. The furthest panel A′ is much larger than the others.The warping is arranged so that each panel subtends the same angularfield of view to the observer (in this case about 19 degrees).

To a first approximation, this distortion can be viewed as projectingthe digital signal from artwork 54 onto a virtual plane 54′ (FIG. 6B),relative to which the observer is on the center axis 84.

FIG. 7 shows the result of this signal warping, in two dimensions. Eachrectangle in FIG. 7 shows the extent of one illustrative signal (e.g.,watermark) tile. Tiles nearest the viewer are relatively smaller; thoseremote are relative larger. (Naturally, in practical application, anartwork will rarely have signal elements of a tiled shape; however, thisassumption facilitates explanation.)

The tile widths shown in FIG. 7 correspond to widths A′-D′ of FIG. 6B.The tile heights also vary in accordance with vertical position of theobserver's perspective (here regarded to be along the vertical mid-lineof the artwork). Tiles near the top and bottom of the artwork are thustaller than tiles along the middle.

When the signal tiles are warped in the FIG. 7 fashion, the signaldetector finds that each tile has substantially the same apparent scale.No longer does a portion of the artwork closer to the observer presentlarger tiles, etc.

Such arrangements allow the artist to author different network linkagesfrom different perspectives. A child viewing a work from a lower heightmay be afforded one set of network features; an adult viewing from agreater height may be linked to others. (In similar fashion, differentvantage points from around a sculpture may lead to different networkresponses.)

In some cases an artist may provide a digital signal at one scale in oneregion of an artwork, and a similar signal, at a different scale, in anadjoining region. In such case, the software tool can include a function(either manually invoked, or automatic) to blend the two regions, andtheir respective signals, so that the transition in signal scale is notvisually apparent.

Some artworks pass through a workflow involving several differentparties. For example, one artist may produce artwork for a magazinecover, working at a magnified scale (e.g., with the artwork three feetin height). This art may be passed to the magazine's art department,which changes the height to eleven inches, and adds graphical elementssuch as the magazine title, issue number, and article teasers. It mayalso slightly adjust the colors in anticipation of the inks used by thepublisher. This revised art may be passed to the printer, which “rips”it—applying parameters specific to the hardware press on which it willbe reproduced. One or more of these further parties in the workflow mayalso be granted permission to elaborate on the artwork's networkextensions. (Post-production workflows in movies and music providemodels for collaborative endeavors that may become common with signalrich art.)

In some cases, artwork having network adjuncts has one of several“looks” by which it can be recognized as having such additionalfeatures—at least when viewed by certain users. Just as different comicstrips have distinctive looks, and anime videos have distinctive looks,styles may emerge by which signal rich art can also beidentified—alerting viewers that associated network functionality isavailable for access. (Alternatively, distinctive logos or icons mayserve this purpose.)

In other cases, artworks may have network adjuncts without advertisingthis fact. Banknotes may be one example. (Banknotes may use a networkcapability for functional and aesthetic roles—enabling transactions,communicating about the national culture, etc.)

The foregoing discussion contemplated that the authoring software alwaysprovides the detailed support for signal rich art. In some arrangements,however, this functionality is a mode that can be enabled or disabled.For novice users, or those with no interest in network aspects of art,the functionality can be turned off. For other users, the signal richmode can be activated.

More Tools

An embodiment incorporating certain principles of the present technologyis a paintbrush. Like most paintbrushes, the present paintbrush includesa body portion and a set of bristles (or other filaments). The bodyportion is typically held by the user. The bristles extend from the bodyof the brush (or, in some implementations, from a ferrule, which isregarded as part of the body) and serve to convey paint or other markingfluid from a source to a writing medium. This fluid source can beexternal, e.g., a palette or an inkwell, or it may comprise part of theimplement—such as an ink reservoir in the paintbrush body.

Included, in a particular embodiment, is a mechanism by which thebristles can be moved relative to the body, in response to electroniccontrol signals. One such mechanism is a piezo-electric actuator, whichapplies a mechanical force in response to applied voltage. This forcecan be used to extend the set of bristles further from the paintbrushbody (or ferrule). (Similarly, the removal of such an applied voltage,or the application of a voltage of opposite polarity, causes the set ofbristles to retract. For expository convenience, all such operations maybe regarded as extending the bristles—the latter simply by a negativeamount.) The lengths of the bristles (or, more accurately, the lengthsof their free portions) are thus varied in accordance with the controlsignal.

A variant embodiment employs a MEMS (microelectromechanical system)device to move the bristles, rather than a piezo-electric unit. Anotherembodiment employs a solenoid with a spring-biased armature. Other formsof actuators can similarly be used.

A controller typically generates the electrical signals to which theactuator responds. This controller can be integrated with the brush(e.g., disposed in the body), or it can be external. An externalcontroller can be coupled to the actuator wirelessly, using knowntechnologies. One particular form of a controller is a digital-to-analogconverter device, which receives control signals in digital form, andoutputs corresponding analog signals to drive the actuator.

A particular brush 62 is depicted schematically in FIGS. 8A and 8B, andincludes a body 64, bristles 66, a ferrule 68, an actuator 70, and acontroller 72. In FIG. 8B, the bristles 64 are relatively extended; inFIG. 8A they are relatively retracted. Typically, the bristles areconfined somewhat snugly by the ferrule 68 (while still permitting theirlinear extension). Thus, only portions 74 of the bristles that extendbeyond the ferrule are free to flex and move in response to the artist'smovement with the brush.

As the artist touches down on the brush, and urges the bristles onto thecanvas or other medium, the bristles naturally spread. In response tosuch action, the relatively extended bristles of the FIG. 8B brush—withtheir longer free portions—spread more broadly than the relativelyretracted bristles of the FIG. 8A brush. This causes the brush of FIG.8B to deposit a broader stroke than the brush of FIG. 8A, when thebrushes are used in the same manner by the artist.

This is shown in FIGS. 9A and 9B. FIG. 9A shows an exemplary brushstroke 76 made without any bristle actuation. FIG. 9B shows, at 78 a,how the width of the stroke enlarges when the bristles 66 are extendedfrom the brush (as in FIG. 8B), allowing them to spread more broadly inresponse to the artist's touch. At 78 b, the bristles are retracted backinto the brush (as in FIG. 8A)—returning the stroke to its former width.The illustrated modulation of line width occurs without any change inpressure by the artist. The artist simply paints as before—the changingof the stroke width occurs independently of any deliberate artistaction.

The magnitude of the broadening is dependent on the artist's brushpressure, and the angle of the brush relative to the medium. Theseparameters may be regarded as the “pose” of the brush. It will berecognized that the width of the stroke is primarily a function of thepose of the brush—as set by the artist, and is only secondarilydependent on modulation due to the actuator.

The amount of stroke broadening shown in FIG. 9B is 30+%. This is largerthan is typical, and is so-depicted simply for clarity of illustration(although in some embodiments there may be broadening of 5%, 10%, 20% ormore). More typically, the broadening is below 5%, with 2% or 3% perhapsbeing typical. Again, however, more extreme cases may have broadening ofless than 2%, 1%, or even 0.5%. Usually, all such encoding is“steganographic” in that it conveys information to a suitably-equippeddetector, but the presence of such encoding is overlooked by casualhuman observers (and may be so minute that it cannot be perceived withthe human visual system even with focused study).

The stroke-width transition at 78 a is rounded. The shape of this roundin a function of the speed at which the bristles extend, and also thespeed at which the artist is moving the brush.

By modulating the width of the brush stroke, digital data can beconveyed. FIG. 10 shows one such arrangement, in which the broadenedareas correspond to dots and dashes of Morse code—here representing theletters DM. A related coding system adapts a 2D barcode symbology, withbroad barcode lines expressed as longer areas of stroke-broadening (likethe ‘dashes’ of FIG. 10), and thin barcode lines expressed as shorterareas of stroke-broadening (like the ‘dots’ of FIG. 10).

While the just-noted coding arrangements use binary symbols (i.e., longand short broadened areas), other arrangement can naturally use three ormore different symbols. Or, forms of unary coding can be employed. Also,in practical implementation, the coding system employed would typicallyinclude built-in redundancy with error correction, to allow reliablycorrect decoding despite different forms of distortion. (Turbo codes,LDPC codes, Reed-Solomon, Viterbi codes, etc., are some codes that maybe used.)

An exemplary decoding system is next described. The artisan willrecognize that a great variety of methods can be employed; the followingis simply a sample.

Decoding starts with capturing a digital image of the artwork. An imageprocessing software program then identifies edges in the digital imageusing known techniques. Examples include the Canny, Differential, andSobel methods. Once edges are identified, the program steps along theidentified edges—computing the tangent or normal function for shortexcerpts (e.g., using the positions of the previous, say, 2, 5, or 15pixels, assuming the analysis is performed in the pixel domain). Thiscomputation is performed with successively overlapping sets of edgepixels, yielding a series of data related to the edge orientation. (Aseries of tangents is shown at 80 in FIG. 10.) The resulting series oftangent or normal function data is examined for change exceeding athreshold (e.g., an orientation change greater than 3, 10 or 30degrees), followed by a return to a value within a first threshold(e.g., 2, 6 or 20 degrees) of the earlier value. Each such event signalsa point at which the contour has ‘jogged’ briefly. Such analysiscontinues—identifying the locations at which the edge shifts.

To determine whether the shift represents a broadening or a narrowing ofthe stroke, the program can perform a similar analysis on nearby edgecontours that are generally parallel to the analyzed edge. Thus, in theFIG. 10 example, the program finds that the bottom edge of the depictedstroke is generally parallel to the edge just-discussed, and examinesshifts in this lower edge. Generally, the detected shifts in the twoedges should exhibit inverse spatial correlation (a measure that canserve as a metric for decoding confidence). By such method, the programidentifies where the stroke is broadened, and where it isnarrowed—thereby yielding the information needed to extract theearlier-encoded information.

The just-detailed method is suitable regardless of whether a dark strokeis painted on a light background, or a light stroke is painted on a darkbackground. If the program has knowledge about whether a stroke isrelatively dark or relatively light, then it can extract the neededinformation for decoding by reference to a single edge contour—withoutlooking for a parallel edge with spatially correlated shifts. That is,if the stroke is dark, then a shift of the contour away from the darkbody of the stroke is known to be a broadening. However, without suchinformation, then a shift of the contour up and away from a dark area(such as on the upper edge of FIG. 10) may be either a broadening of adark stroke (located below), or a narrowing of a light stroke (locatedabove).

In the examples just-given, correct decoding requires knowledge of thecoding direction. In the depicted Morse code arrangement, the codeadvances with the artist's stroke. But a later decoder does not knowwhether a subject stroke was painted from left-to-right, or fromright-to-left. Thus, desirably the coding includes an indication aboutdirection. A simple indication is the periodic use of a symbol that hasno valid inverse (e.g., the letter Z in Morse is dash-dash-dot-dot;Morse has no dot-dot-dash-dash symbol). When such a symbol is found, thecorrect decoding direction is thereby indicated.

A related issue concerns clocking. In some embodiments (but not all) thebroadening and narrowing of brush strokes occur at temporally-definedintervals, e.g., every tenth or half second. Depending on the speed withwhich the artist moves the brush, a “dash” in the FIG. 10 example may bean inch long, or a small fraction thereof. A decoder encountering aseries of quarter-inch broadened regions may not know whether theyrepresent a series of dashes or a series of dots. Desirably a symbol isinserted periodically by which the decoder can use as a reference indetection. Again, the Morse letter Z is a simple example, as it includesboth dashes and dots. By providing such a symbol periodically, thedecoder has a reference by which it can distinguish dashes and dots.

As just-suggested, the broadening and narrowing of brush strokes needn'tbe temporally-based. Instead, these variations can be spatially-based.For example, the extent of a dash may be set in space (e.g., ahalf-inch) rather than in time (e.g., a half-second). So-doing requiresknowledge of the brush's location on the medium.

There are many techniques known for position determination; virtuallyany can be used. One is to position the medium on a platen that sensesthe X-, Y-position of the writing implement, such as by pressure,capacitance, electromagnetic induction, etc. Particular examples includethe graphics tablets manufactured by Wacom (e.g., the Intuos 4, andBamboo Pen), in which the writing implement generates an electromagneticsignal that is sensed by a grid of wires in the underlying platen (or,reciprocally, in which the grid of wires emit electromagnetic signalsthat are detected by a sensor in the implement). In such embodiments,power for the circuitry in the writing implement can be derived, e.g.,inductively, from the platen below.

Other position-sensing arrangements are acoustic in nature, with thewriting implement issuing ultrasonic clicks that are sensed bymicrophones near the periphery of the medium. Time-of-acoustic-travel isused to locate the writing implement in X- and Y-.

Optical techniques can also be used. Such arrangements employ a smallcamera in the writing implement to sense features on the medium that arecorrelated with location. (In some embodiments the paper is lightlymarked with dots that encode position data.) Examples are shown inAnoto's U.S. Pat. Nos. 5,477,012, 6,570,104 and 7,588,191, and inSilverbrook Research's U.S. Pat. Nos. 6,870,966 and 7,396,177.

GPS, in its current form, is not well suited to the present application,due to the relative coarseness of location information. However, otherradio-based location technologies can be employed. One type utilizesradio signals of the sort that are that commonly exchanged betweendevices (e.g., WiFi, cellular, etc.). Given several communicatingdevices, the signals themselves—and the imperfect digital clock signalsthat control them—form a reference system from which both highlyaccurate time and position can be abstracted. Such technology isdetailed in published patent applications 2009213828, 2009233621,2009313370, 2010045531, and 2010202300. A writing implement can beequipped with a transmitter, and two or more other stations cancommunicate with it to thereby determine the implement's location.

Yet another position sensing system relies on accelerometers orgyroscopic devices within the brush, to track its movement over time. (AMEMs accelerometer that can be used in certain embodiments is theLIS302DL, a three-axis device manufactured by STMicroelectronics. Asuitable 3-axis gyroscope is the STMicroelectronics L3G4200D.) A varietyof known software can take input data from such a device, and discern—inrelative spatial terms—how the device moves in space and time.

In embodiments in which the available position data is from a locationother than the tip of the implement (e.g., from a sensing device in thebody of a brush), an algorithm can be applied to re-map the originalsensed data to deduce the corresponding position at the tip of theimplement.

In still other arrangements, the position/movement of the user's handmay be tracked (e.g., by a MEMs or radio device in a ring worn by theuser) rather than the position/movement of the implement, per se.(Again, the position/movement data from such system can be processed todeduce the position/movement at the tip of the implement.)

FIG. 12 shows an illustrative system using a position-based encodingarrangement 50. A module 51 determines position of the brush using oneof the foregoing techniques. A memory 53 provides a data payload that isto be encoded into the artwork. A processor 52 notes where the brush ison the medium and, based on this data and on the information to beencoded, sends data to the brush actuator—through a wireless interface54—instructing whether the bristles should be extended. As the artistmoves the brush to different positions on the canvas, the processor 52sends updates to the actuator, so that in some locations the bristlesare extended, and in other locations they are not. The edges of thebrush stroke contour are dynamically tweaked in position, in accordancewith brush location.

(In some embodiments, such as those in which the brush senses its ownlocation, a back-channel can be provided through which the brush cantransmit data to the wireless interface 54.)

While the arrangements previously-described may be regarded as usingone-dimensional encoding (e.g., along the contour of a stroke), otherembodiments can employ two-dimensional encoding. An example oftwo-dimensional encoding is the digital watermarking technology detailedin Digimarc's U.S. Pat. Nos. 6,345,104, 6,449,377, 6,590,996 and7,286,684.

In one particular such embodiment, the artwork canvas is virtuallydivided into a grid of 2D tiles, each about 5-50 cm on a side. Each ofthese tiles, in turn, is virtually divided into an array of 128×128“waxels” (“watermark elements”). The luminance of each of these waxels,relative to its four or eight nearest neighbors (c.f., Digimarc's U.S.Pat. Nos. 6,483,927, 6,580,809 and 7,231,061), is adjusted slightly upor down to effect encoding of a pattern that conveys the desired messagepayload. The pattern can also include a calibration signal to aid inextraction of the encoded message payload (as further detailed in thejust-cited patents).

In another particular embodiment, the artwork defines a singlewatermarking tile; there is no tiled redundancy.

In some such embodiments, the brush may have two or more actuators, bywhich the stroke width can be bulged slightly on one side of the brush(e.g., by extending the length of those filaments) without changing thecontour on the other side. The position information output by module 51can indicate not only the X-Y position of the brush, but also itsspatial orientation, allowing the two actuators to be controlledindependently to change the two contours of the stroke independently. Bysuch arrangement, a brush stroke contour can be slightly adjusted todeposit more, or less, paint in a given waxel, under control of theprocessor 52.

The processor 52 can consider not just the brush's present location, butalso the direction in which it is traveling, and send instructions tothe brush actuator(s) in anticipation of where the brush appears to beheaded. This is desirable because there is a small, but detectable,interval between when instructions to extend bristles are sent to thebrush, and when the lengthened bristles actually spread wider under theartist's touch to start depositing a broader stroke.

A reference frame for the watermark (e.g., its origin point, andorientation) may be established by the artist's initial use of a cleanbrush, or other drawing implement. In an exemplary arrangement theartist touches three locations on the media in rapid succession (e.g.,within a threshold interval such as 1-4 seconds). An accelerometer inthe instrument communicates the three taps or touches (i.e., threede-accelerations exceeding a threshold value) to associated watermarkencoding software. (The spatial locations of the taps are alsodetermined by means described below.) These taps indicate to thesoftware that the drawing device is ready for action, and the threepoints in space where the drawing surface was touched define atwo-dimensional plane in 3D space. The first touched point canarbitrarily become the origin of a watermark grid system, and the linebetween the origin and the second tapped point can become the X- orY-axis of the coordinate system in 3D space. (The MEMs device itself andits associated software usually employs a different, relatively random3D origin for its own tracking of 3-dimensional movement, e.g., thedevice position when it was first powered, or most recently reset.)

In another arrangement, the spatial definition of the 2D drawing planecan be sleuthed by simply having a person draw something over a fewseconds' interval, or write a quick word or two, etc. The positionsindicated by the system sensors define the plane of the work. Theinitial point of contact can define the origin, and the direction ofinitial implement movement can define a reference axis.

In still other embodiments, the origin and a reference axis can beestablished randomly, or by some other algorithm, once the plane of thework is known.

The watermark encoding software chooses a characteristic waxel size(which may typically be on the order of 1-5 mm or so for something likea magic marker), and proceeds to virtually map out an embedding template(e.g., 128×128 waxels) within the newly defined 2D plane within 3Dspace. The watermark embedding software monitors the ongoing progress ofthe “tip” of the drawing device, and when it sees that the tip isgetting to within a couple centimeters of the virtual 2D watermarkingplane, it begins to send control signals to the implement. Then, as thetip of the drawing device moves from region to region within the 2Dwatermarking plane, instructions are sent to the implement to modulateits output according to the values contained in the 2D watermarkembedding template. It may be noted that this is a “differentialembedding” approach, as opposed to a straight-forward embeddingapproach. That is, in order to create a “darker waxel” next to a“lighter waxel,” the watermarking software needs to also understandspatial trajectories of the tip, and as it moves toward a “darkerwaxel,” it needs to jiggle bristles toward that region as it approachesthe region, thus applying more paint there. Then, as it crosses over theregion and moves on, the instruction to the actuator may cause thebristles to extend back towards the darker region, thus continuing totry to apply more paint to the area on the drawing surface calling for adarker waxel.

A watermarked artwork may be decoded using a mobile phone with suitabledecoding software. The software may expect sparsely-covered watermarksource material, such as handwriting—in which case the captured imagedata may be filtered, or otherwise pre-processed, so that pixel regionsthat probably haven't been affected by the watermark signal arediscarded. (For example, in embodiments in which watermarking iseffected by modulating the edges of brush strokes, a region of apainting that is devoid of edges, e.g., a blue sky, may be disregardedin a watermark decoding operation.) Conversely, the software can analyzecaptured image data to identify regions that seem most-likely to conveywatermark data, and focus its decoding effort on such regions. U.S. Pat.Nos. 6,442,284, 6,516,079 and 7,013,021 detail techniques of this sort.

As noted in the cited patents, the watermark decoder can combinecorresponding waxels from several tiles to increase signal-to-noiseratio before applying a decoding algorithm.

Having described and illustrated the principles of our technology withreference to various embodiments thereof, it should be apparent that theinvention can be modified in arrangement and detail without departingfrom such principles.

For example, while the earlier-detailed brush 12 extends the bristlesgenerally along the axis of the paintbrush body, another arrangementcauses the bristles to spread out—away from the axis of the body (insome cases radially outwardly). One such arrangement is shown in thebrushes 90 that are depicted, in section, in FIGS. 11A and 11B. Here, adispersing member 92 (e.g., a hollow cylinder) is formed within theferrule. As some of the bristles are forced towards the dispersingmember 92 by the actuator, they are deflected outwardly. This causes thebristles to fan out, to a degree dependent on the actuator controlsignal. Thus, the fan width 94 b of the brush with extended bristles isgreater than the fan width 94 a of the brush with retracted bristles.(The bristles are shown lighter in FIGS. 11A and 11B to better show thedispersing member 92.)

In the above-described embodiments, the bristles are pushed—orpulled—from their ends by an actuator. In other arrangements, anactuator can otherwise cause deflection or other movement of thebristles. For example, the bristles may be fastened to a stationarymember in the brush body, and an actuator can act at an intermediatelocation along their length to bias the bristles in one direction oranother.

Similarly, while the discussed embodiments each employed a singleactuator, in other embodiments, plural actuators can be used. Forexample, two or three actuators can be provided, e.g., one to controlposition of the bristle tips in an X-direction, and/or one to controlposition of the tips in a Y-direction, and/or one to control position ofthe tips in a Z-direction.

While linear actuators have been particularly noted, other types ofactuators (e.g., rotary) can naturally be employed. Likewise, membersthat move in response to magnetic, electrostatic, or other stimulus canbe employed with similar effect.

The brush bristles can be of any desired type, e.g., sable, nylon, etc.

While the detailed embodiment particularly considered modulating thewidth of strokes to convey data (i.e., changing luminance), this is butone of many different parameters that may be changed to effect dataencoding. A few others include saturation, density, color, texture andpattern. From the teachings herein, an artisan can adapt encoding anddecoding methods based on variations in one or more such parameters.

It will be recognized that a brush is but one type of writing implementthat can utilize the concepts detailed herein. Pencils, crayons and inkpens are others. A particular type of ink pen that offers digitalcontrol of various stroke parameters is detailed in U.S. Pat. No.7,396,177 (attached hereto as Appendix A, and forming part of thisspecification).

Although described in the context of a brush that deposits ink on aphysical medium, the same principles are applicable to digital artcreation tools—such as using a stylus on a drawing tablet usingPhotoshop.

Related technologies are detailed in the following patent publications:20040189620, 20050123340, 20070139399, 20080055279, 20080152202,20090267896, U.S. Pat. Nos. 5,215,397, 5,247,137, 5,294,792, 5,434,371,5,781,661, 5,781,661, 5,981,883, 6,573,887, 6,698,660, 7,773,076,7,796,819, and WO2006068357.

Intuitive computing technologies, of which the present technology may beregarded as an example, are detailed in Digimarc's published PCTapplication WO2010022185 and in pending application Ser. No. 12/797,503,filed Jun. 9, 2010 (published as 20110161076).

Other Comments

While the specification has made repeated reference to elements such assmartphones and graphical user interfaces, it will be recognized thatmany of the detailed elements will be superseded in years to come.Head-worn display devices with integrated cameras (e.g., the Vuzix iWearVR920), and gestural interfaces (e.g., Microsoft's Kinect), for example,are already appearing. Thus, it should be recognized that the detailedarrangements are illustrative, and not meant to limit the forms in whichthe detailed technology can be practiced.

Steganography and digital watermarking techniques, and applicationsresponsive to digital signals in content (e.g., “Mediabridge”), aredetailed in Digimarc patents including U.S. Pat. Nos. 6,122,403,6,590,996, and 6,947,571, and published application 20060115110.

Image and video fingerprinting techniques are detailed in patentpublications U.S. Pat. No. 7,020,304 (Digimarc), U.S. Pat. No. 7,486,827(Seiko-Epson), 20070253594 (Vobile), 20080317278 (Thomson), and20020044659 (NEC).

Information about palettes, brushes and other aspects of the Photoshopuser interface are detailed in Adobe's patent literature, including U.S.Pat. Nos. 4,837,613, 5,592,602, 5,870,091, 6,606,105, 6,606,166,20020130908, 20020150307, 20040246271, and 2005001854.

The design of smartphones and computer devices referenced in thisdisclosure is familiar to the artisan. In general terms, each includesone or more processors (e.g., of an Intel, AMD or ARM variety), one ormore memories (e.g. RAM), storage (e.g., a disk or flash memory), a userinterface (which may include, e.g., a keypad, a TFT LCD or OLED displayscreen, touch or other gesture sensors, a camera or other opticalsensor, a compass sensor, a 3D magnetometer, a 3-axis accelerometer, amicrophone, etc., together with software instructions for providing agraphical user interface), interconnections between these elements(e.g., buses), and an interface for communicating with other devices(which may be wireless, such as GSM, CDMA, W-CDMA, CDMA2000, TDMA,EV-DO, HSDPA, WiFi, WiMax, mesh networks, Zigbee and other 802.15arrangements, or Bluetooth, and/or wired, such as through an Ethernetlocal area network, a T-1 internet connection, etc).

More generally, the processes and system components detailed in thisspecification may be implemented as instructions for computing devices,including general purpose processor instructions for a variety ofprogrammable processors, including microprocessors, graphics processingunits (GPUs, such as the nVidia Tegra APX 2600), digital signalprocessors (e.g., the Texas Instruments TMS320 series devices), etc.These instructions may be implemented as software, firmware, etc. Theseinstructions can also be implemented in various forms of processorcircuitry, including programmable logic devices, FPGAs (e.g., XilinxVirtex series devices), FPOAs (e.g., PicoChip brand devices), andapplication specific circuits—including digital, analog and mixedanalog/digital circuitry. Execution of the instructions can bedistributed among processors and/or made parallel across processorswithin a device or across a network of devices. Transformation ofcontent signal data may also be distributed among different processorand memory devices.

Software instructions for implementing the detailed functionality can bereadily authored by artisans, from the descriptions provided herein,e.g., written in C, C++, Visual Basic, Java, Python, Tcl, Perl, Scheme,Ruby, etc. Known image editing software, such as Adobe's Creative Suite(including Photoshop) and GIMP, and known augmented reality software,such as UrbanSpoon, Layar, Bionic Eye, Wikitude, Tonchidot, etc., can beadapted for many of the uses detailed herein. (GIMP is image editingsoftware similar to Photoshop, but for which source code is availablefrom the gimp<dot>org web site.)

Commonly, each device includes operating system software that providesinterfaces to hardware resources and general purpose functions, and alsoincludes application software which can be selectively invoked toperform particular tasks desired by a user.

Software and hardware configuration data/instructions are commonlystored as instructions in one or more data structures conveyed bytangible media, such as magnetic or optical discs, memory cards, ROM,etc., which may be accessed across a network. Some embodiments may beimplemented as embedded systems—a special purpose computer system inwhich the operating system software and the application software isindistinguishable to the user (e.g., as is commonly the case in basiccell phones). The functionality detailed in this specification can beimplemented in operating system software, application software and/or asembedded system software.

Any of the processing entailed in embodiments incorporating the presenttechnology can be performed “in the cloud.”

U.S. Pat. No. 6,577,746 notes that a photocopier may identify an imagein a scanned paper document by reference to embedded watermark data, andcan then use this identification to download a pristine version of thesame image for substitution into photocopied output, yielding a “copy”that may be better than the original. Relatedly, imaging devices such assmart phones may recognize imagery, such as elements of artwork, anddownload versions of the same or related content for viewing or otheruse. (A tourist may take a snapshot of the Eiffel Tower. By featurerecognition such as SIFT or image fingerprinting, the camera may be ableto identify the subject of the photo, and perhaps the viewing pose, andthen access a large archive of related imagery. The camera may thenquery the user whether he would like the camera to obtain a highresolution view of the scene suitable for framing, a nighttime shot, asunset depiction, etc., etc.)

As will be recognized, the present specification has detailed many novelarrangements. Due to practical constraints, many such arrangements arenot claimed in the original filing of this application, yet applicantsintend to claim such other subject matter in subsequent applicationsclaiming priority. An incomplete sampling of some of the inventivearrangements is reviewed in the following paragraphs:

An arrangement including program instructions stored on a computerreadable medium, and operative to configure a programmable processingsystem to perform acts including (a) provide plural authoring tools thatenable a user to define elements of a work of art, at least one of saidtools providing a network hook by which network functionality can beassociated with an element; and (b) provide at least one networkauthoring tool that enables a user to define network functionalityassociated with one or more of said network hooks. By such arrangement,the configured system is adapted to produce artwork comprised ofelements, at least certain of which are associated with user-definednetwork functionality.

An arrangement for authoring an artwork, that includes providing (e.g.,in a graphical user interface) a first tool configured to be manipulatedby a user to define one or more elements of an artwork, such one or moreelements comprising a network hook by which network functionality can beassociated with the artwork; and also providing a second tool thatenables a user to define network functionality associated with one ormore of said network hooks. By such arrangement, artwork can be producedassociated with user-defined network functionality.

Another arrangement is an authoring system that includes a processor, amemory and a display, the memory including instructions that program theprocessor to present a user interface that includes, on the display: (a)first portion from which a user can select a portion of an artwork, and(b) a second portion from which the user can select networkfunctionality to be associated with said selected portion.

A further arrangement includes receiving data corresponding to a user'sfirst art stroke, and establishing (in an artwork) a first stroke regionthat includes a first digital signal; then receiving data correspondingto a user's second art stroke, and establishing in the artwork a secondstroke region that includes a second digital signal; and then storinginformation in a data repository, where the stored informationassociates a first network function with the first stroke region of theartwork, and associates a second network function with the second strokeregion of the artwork.

Another aspect is an arrangement including: receiving, through a userinterface, user instructions defining an arrangement of elements to forma work of artwork, where the artwork including a representation of aface; and then—through use of a hardware processor—modifying therepresentation of a face to convey a digital signal representing pluralsymbols.

A still further aspect is a work of visual art presented on a tangibledisplay or medium, where the work of art includes a digital signal thatis associated with auxiliary information (e.g., audio information) forpresentation to a viewer, and where the association changes over time.Thus, a viewer at a first time is presented with first auxiliaryinformation, and a viewer at a later time is presented with second,different, auxiliary information.

A related aspect is a work of visual art that includes a digital signalthat is associated with auxiliary information for presentation to aviewer, wherein the association varies with different viewers. Thus, afirst viewer is presented with first auxiliary information, and a secondviewer is presented with second, different, auxiliary information.

Another related aspect is a work of visual art having a first regionincluding a first digital signal that is associated with first audioinformation, and having a second region including a second digitalsignal that is associated with second audio information different thanthe first. Thus, a viewer can be presented with the first or the secondaudio information, depending on a region of the art with which theviewer interacts.

Another arrangement is a writing apparatus that includes a body, anactuator, and plural filaments extending from the body. The actuator isadapted to move the filaments relative to the body, in response toelectronic control signals.

Another of the detailed arrangements is a method that includesdetermining a spatial position of a writing implement over a writingmedium; and discharging a writing fluid from the writing implement ontothe writing medium, to form a written stroke. A parameter of the writtenstroke—comprising at least one of saturation, density, thickness, color,or pattern—is controlled both by a pose of the writing implementrelative to the medium, and, to a lesser extent, by said determinedspatial position of the writing implement.

Another arrangement is a method that includes sensing touches of animplement at first, second and third spaced-apart positions on asurface, and determining a 2D plane in 3D space by reference tolocations of said touches. The method further includes defining areference axis by reference to locations of two of said touches, anddefining an origin of a coordinate system in the plane by reference toone of the touches. Such operations establish a virtual coordinateframework by reference to which strokes comprising artwork can belocated.

To provide a comprehensive disclosure without unduly lengthening thisspecification, applicants incorporate by reference the patents and otherdocuments referenced herein. (Such references are incorporated in theirentireties, even if cited above in connection with specific of theirteachings.) These references disclose technologies and teachings thatcan be incorporated into the arrangements detailed herein, and intowhich the technologies and teachings detailed herein can beincorporated.

1-22. (canceled)
 23. A method of providing augmentation data tocamera-equipped user devices, the method comprising the acts: providingartwork for a first printed item, the artwork including amachine-readable code, the machine-readable code encoding an identifier;creating an augmentation to be presented on a display of a user device,when the user device captures imagery of said first printed item, andstoring augmentation data defining said augmentation, in associationwith said identifier; in connection with capture of imagery of saidfirst printed item by a first user device, (a) logging, in a databaserecord, data associating the first user device and the first printeditem, indicating the first user device has interacted with the firstprinted item; and (b) sending the stored augmentation data to the firstuser device, for presentation of the augmentation on a display of saidfirst user device; and in connection with capture of imagery of saidfirst printed item by a second user device, (a) logging, in a databaserecord, data associating the second user device and the first printeditem, indicating the second user device has interacted with the firstprinted item; (b) determining, from previously-logged data, that thesecond user device has previously interacted with different printeditems, including different machine readable codes; and (c) based on saiddetermination, sending data to the second user device that is differentthan the data sent to the first device.
 24. The method of claim 23 inwhich the augmentation comprises an augmentation overlay that is to bepresented on a display of a camera-equipped user device, together withimagery captured by the camera of said device.
 25. A method forauthoring product packaging label artwork, including aprocessor-controlled hardware system performing the acts: providing agraphical user interface that includes authoring tools enabling a userto edit layered elements of product packaging label artwork, saidlayered elements including first and second layered elements; receivinga first identifier; steganographically-encoding the first identifier insaid first layered element, but not in said second layered element;determining how the steganographic encoding of the product packaginglabel artwork will be transformed by a rendering operation, saiddetermining comprising assessing relative strength of the steganographicencoding after such rendering operation; presenting results of saidassessment for user review, in the form of a 2D strength map; and aftersaid analyzing, revising the steganographic encoding of said firstidentifier in said first product packaging label artwork layeredelements.
 26. The method of claim 25 in which the determining actincludes rendering the product packaging label artwork.
 27. The methodof claim 25 in which the determining act includes modeling the renderingof the product packaging label artwork.
 28. The method of claim 25 inwhich the steganographic encoding comprises subtracting an encodingpattern from the first layered elements.
 29. The method of claim 25 inwhich the revising comprises a software tool responding to said act ofassessing by automatically adjusting the artwork to enhancerecognizability of the steganographic encoding.
 30. The method of claim25 wherein the product packaging label artwork also includes a secondproduct packaging label artwork element that conveys an identifier,wherein detection of the first product packaging label artwork elementat a first region by a consumer's mobile device leads to presentation ofa first network response to the consumer, and detection of the secondproduct packaging label artwork element at a second region by theconsumer's mobile device leads to presentation of a second, different,network response to the consumer.
 31. The method of claim 30 in whichthe second identifier is different than the first identifier.
 32. Themethod of claim 30 in which the second product packaging label artworkelement comprises a 1D or 2D barcode.
 33. The method of claim 25 thatfurther includes: creating an augmented reality overlay to be presentedon a display of a consumer's mobile device in response to interactionbetween the mobile device and the product packaging label artwork;storing augmentation data defining said augmented reality overlay, inassociation with said identifier; in connection with capture of imageryof said product packaging label artwork by the consumer's mobile device,and decoding of the identifier from the steganographically encodedartwork element, (a) logging, in a database record, data associating theconsumer or the consumer's mobile device with the product packaginglabel artwork, indicating the consumer has interacted with the productpackaging label artwork; and (b) sending the stored augmentation data tothe consumer's mobile device, for presentation of the augmented realityoverlay.
 34. The method of claim 33 that further includes, in connectionwith capture of imagery of the product packaging label artwork by asecond consumer's mobile device, (a) logging, in a database record, dataassociating the second consumer or the second consumer's mobile devicewith the product packaging label artwork, indicating the second consumerhas interacted with the product packaging label artwork; (b)determining, from previously-logged data, that the second consumer haspreviously interacted with different artwork, which steganographicallyencoded a different identifier; and (c) based on said determination,sending data to the second consumer's mobile device that is differentthan the stored augmentation data sent to the first consumer's mobiledevice.
 35. The method of claim 25 in which the first layered element isadded to the packaging label artwork by a user employing said graphicaluser interface, but the steganographic encoding of said first layeredelement with the received identifier is not performed until after thefirst layered element is added.
 36. The method of claim 35 in whichsteganographic encoding of the first layered element is simulated bytexturing that is displayed on a computer screen presented to the userto represent the label artwork.
 37. The method of claim 35 in which thesteganographic encoding of the first layered element is performed afterthe user's creative process has concluded.